A Simple Ensemble Method for Hedge Identification

We present in this paper a simple hedge identification method and its application on biomedical text. The problem at hand is a subtask of CoNLL-2010 shared task. Our solution consists of two classifiers, a statistical one and a CRF model, and a simple combination schema that combines their predictions. We report in detail on each component of our system and discuss the results. We also show that a more sophisticated combination schema could improve the F-score significantly. 1 Problem definition The CoNLL-2010 Shared Task focused on the identification and localization of uncertain information and its scope in text. In the first task, a binary classification of sentences had to be performed, based on whether they are uncertain or not. The second task concentrated on the identification of the source of uncertainty – specifying the keyword/phrase that makes its context uncertain –, and the localization of its scope. The organizers provided training data from two application domains: biomedical texts and Wikipedia articles. For more details see the overview paper by the organizers (Farkas et al., 2010). We focused on task 1 and worked with biomedical texts exclusively. The biomedical training corpus contains selected abstracts and full text articles from the BioScope corpus (Vincze et al., 2008). The corpus was manually annotated for hedge cues on the phrase level. Sentences containing at least one cue are considered as uncertain, while sentences with no cues are considered as factual. Though cue tagging was given in the training data, their marking in the submission was not mandatory. The evaluation of systems at task 1 was performed on the sentence level with the F-measure of the uncertain class being the official evaluation metric. For evaluation, corpora also from both domains were provided that allowed for in-domain and cross-domain experiments as well. Nevertheless, we restricted the scope of our system to the in-domain biomedical subtask.